Traffic signal response for autonomous vehicles

ABSTRACT

Aspects of the disclosure relate to determining whether a vehicle should continue through an intersection. For example, the one or more of the vehicle&#39;s computers may identify a time when the traffic signal light will turn from yellow to red. The one or more computers may also estimate a location of a vehicle at the time when the traffic signal light will turn from yellow to red. A starting point of the intersection may be identified. Based on whether the estimated location of the vehicle is at least a threshold distance past the starting point at the time when the traffic signal light will turn from yellow to red, the computers can determine whether the vehicle should continue through the intersection.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/958,365, filed on Apr. 20, 2018, which is a continuation ofU.S. patent application Ser. No. 15/618,461, filed on Jun. 9, 2017, nowU.S. Pat. No. 10,005,460, issued on Jun. 26, 2018, which is acontinuation of U.S. patent application Ser. No. 14/448,299, filed onJul. 31, 2014, now U.S. Pat. No. 9,707,960, issued on Jul. 18, 2017, thedisclosures of which are hereby incorporated herein by reference.

BACKGROUND

Autonomous vehicles, such as vehicles which do not require a humandriver, may be used to aid in the transport of passengers or items fromone location to another. An important component of an autonomous vehicleis the perception system, which allows the vehicle to perceive andinterpret its surroundings using cameras, radar, sensors, and othersimilar devices. The perception system executes numerous decisions whilethe autonomous vehicle is in motion, such as speeding up, slowing down,stopping, turning, etc. Autonomous vehicles may also use the cameras,sensors, and global positioning devices to gather and interpret imagesand sensor data about its surrounding environment, e.g., parked cars,trees, buildings, etc.

Information from the perception system may be combined with highlydetailed map information in order to allow a vehicle's computer tosafely maneuver the vehicle in various environments. This highlydetailed map information may describe expected conditions of thevehicle's environment such as the shape and location of roads, trafficsignals, and other objects. In this regard, the information from theperception system and detailed map information may be used to assist avehicle's computer in making driving decisions involving intersectionsand traffic signals.

BRIEF SUMMARY

One aspect of the disclosure provides a method. The method includesidentifying, by one or more computing devices, a time when the trafficsignal light will turn from yellow to red; estimating, by the one ormore computing devices, a location of a vehicle at the estimated timethat the traffic signal light will turn from yellow to red; identifying,by the one or more computing devices, a starting point of anintersection associated with the traffic signal light; determining, bythe one or more computing devices, whether the estimated location of thevehicle will be at least a threshold distance past the starting point;and when the estimated location of the vehicle is determined to be atleast a threshold distance past the starting point, determining, by theone or more computing devices, that the vehicle should continue throughthe intersection.

In one example, the method also includes, when the estimated location ofthe vehicle is determined not to be at least a threshold distance pastthe starting point, determining that the vehicle should stop at orbefore the starting point. In another example, the method also includesidentify a time when a traffic signal light turned from green to yellow,identifying a length of time that the traffic signal light will remainyellow, and identifying the time when the traffic signal light will turnfrom yellow to red is based on the time when a traffic signal lightturned from green to yellow and a length of time that the traffic signallight will remain yellow. In this example, identifying the length oftime include calculating the length of time based on a speed limit of aroadway on which the vehicle is traveling. Alternatively, identifyingthe length of time includes retrieving a default value. In anotheralternative, identifying the length of time includes accessing aplurality of lengths of times for various traffic signal lights andretrieving a value corresponding to the traffic signal light. In anotherexample, the method also includes, after determining that the vehicleshould go through the intersection, estimating, a second location of thevehicle at the time that the traffic signal will turn red, anddetermining whether the estimated second location of the vehicle will beat least a second threshold distance past the starting point. In thisexample, the second threshold distance is less than the thresholddistance. In another example, the estimated location of the vehiclecorresponds to an estimated location of a portion of the vehicleproximate to the rear of the vehicle. In another example, identifyingthe time when the traffic signal light will turn from yellow to red isbased on information received from a device remote from the vehicle.

Another aspect of the disclosure includes a system having one or morecomputing devices. The one or more computing devices are configured toidentify a time when the traffic signal light will turn from yellow tored; estimate a location of a vehicle at the estimated time that thetraffic signal light will turn from yellow to red; identify a startingpoint of an intersection associated with the traffic signal light;determine whether the estimated location of the vehicle will be at leasta threshold distance past the starting point; and when the estimatedlocation of the vehicle is determined to be at least a thresholddistance past the starting point, determine that the vehicle shouldcontinue through the intersection.

In one example, the one or more computing devices are also configuredto, when the estimated location of the vehicle is determined not to beat least a threshold distance past the starting point, determine thatthe vehicle should stop at or before the starting point. In anotherexample, the one or more computing devices are also configured toidentify a time when a traffic signal light turned from green to yellow;identify a length of time that the traffic signal light will remainyellow; and identify the time when the traffic signal light will turnfrom yellow to red based on the time when a traffic signal light turnedfrom green to yellow and a length of time that the traffic signal lightwill remain yellow. In this example, the one or more computing devicesare further configured to identify the length of time by calculating thelength of time based on a speed limit of a roadway on which the vehicleis traveling. Alternatively, the one or more computing devices are alsoconfigured to identify the length of time by retrieving a default value.In another alternative, the one or more computing devices are alsoconfigured to identify the length of time by accessing a plurality oflengths of times for various traffic signal lights and retrieving avalue corresponding to the traffic signal light. In another example, theone or more computing devices are also configured to, after determiningthat the vehicle should go through the intersection, estimate, a secondlocation of the vehicle at the time that the traffic signal will turnred and determine whether the estimated second location of the vehiclewill be at least a second threshold distance past the starting point. Inthis example, the second threshold distance is less than the thresholddistance. In another example, the estimated location of the vehiclecorresponds to an estimated location of a portion of the vehicleproximate to the rear of the vehicle. In another example, the one ormore computing devices are further configured to identify the time whenthe traffic signal light will turn from yellow to red based oninformation received from a device remote from the vehicle.

A further aspect of the disclosure provides a non-transitory, tangiblecomputer-readable storage medium on which computer readable instructionsof a program are stored. The instructions, when executed by one or moreprocessors, cause the one or more processors to perform a method. Themethod includes identifying a time when the traffic signal light willturn from yellow to red; estimating a location of a vehicle at theestimated time that the traffic signal light will turn from yellow tored; identifying a starting point of an intersection associated with thetraffic signal light; determining whether the estimated location of thevehicle will be at least a threshold distance past the starting point;and when the estimated location of the vehicle is determined to be atleast a threshold distance past the starting point, determining that thevehicle should continue through the intersection.

In one example, the method also includes, when the estimated location ofthe vehicle is determined not to be at least a threshold distance pastthe starting point, determining that the vehicle should stop at orbefore the starting point. In another example, the method also includesidentify a time when a traffic signal light turned from green to yellow,identifying a length of time that the traffic signal light will remainyellow, and identifying the time when the traffic signal light will turnfrom yellow to red is based on the time when a traffic signal lightturned from green to yellow and a length of time that the traffic signallight will remain yellow. In this example, identifying the length oftime include calculating the length of time based on a speed limit of aroadway on which the vehicle is traveling. Alternatively, identifyingthe length of time includes retrieving a default value. In anotheralternative, identifying the length of time includes accessing aplurality of lengths of times for various traffic signal lights andretrieving a value corresponding to the traffic signal light. In anotherexample, the method also includes, after determining that the vehicleshould go through the intersection, estimating, a second location of thevehicle at the time that the traffic signal will turn red, anddetermining whether the estimated second location of the vehicle will beat least a second threshold distance past the starting point. In thisexample, the second threshold distance is less than the thresholddistance. In another example, the estimated location of the vehiclecorresponds to an estimated location of a portion of the vehicleproximate to the rear of the vehicle. In another example, identifyingthe time when the traffic signal light will turn from yellow to red isbased on information received from a device remote from the vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional diagram of a system in accordance with aspects ofthe disclosure.

FIG. 2 is an interior of an autonomous vehicle in accordance withaspects of the disclosure.

FIG. 3 is an example of detailed map information in accordance withaspects of the disclosure.

FIG. 4 is an exterior of an autonomous vehicle in accordance withaspects of the disclosure.

FIG. 5 is an example diagram of a section of roadway including anintersection in accordance with aspects of the disclosure.

FIG. 6 is an example diagram of an expected location of a vehicle inaccordance with aspects of the disclosure.

FIG. 7 is another example of detailed map information in accordance withaspects of the disclosure.

FIG. 8 is an example of detailed map information combined with anexample diagram of a section of roadway including an intersection inaccordance with aspects of the disclosure.

FIG. 9 is another example of detailed map information combined with anexample diagram of a section of roadway including an intersection inaccordance with aspects of the disclosure.

FIG. 10 is an example flow diagram in accordance with aspects of thedisclosure.

DETAILED DESCRIPTION OVERVIEW

The technology relates to determining whether an autonomous vehicle, ora vehicle driving in an autonomous mode, should proceed through or stopin response to a yellow traffic light. The case of a yellow light can beespecially difficult to address. For example, simply braking for allyellow lights may not be a good decision; if there are other vehicles inthe vicinity, in particular those behind the autonomous vehicle, suchother vehicles may not expect a sudden stop when there is still time topass through the light. This could then lead to accidents. Other factorswhich can complicate determining a response may include whether thereare other vehicles in front of the autonomous vehicle which are movingslowly, but have already reached an intersection when the traffic signalturns yellow. In such a case, proceeding through the intersection maycause the autonomous vehicle to drive into the intersection when thetraffic signal is actually red. Accordingly, the features describedherein address how a computer can determine whether the autonomousvehicle should continue through a traffic intersection when a trafficsignal has turned red.

An autonomous vehicle's one or more computing devices may identify thestate of a traffic signal using any known techniques. For example, usinga combination of sensor data and detailed map information, the computersmay estimate an approximate location of a traffic signal. Then usingtemplates, image matching color detection in images, etc. the computersmay determine the state of a traffic signal (red, yellow, or green).Alternatively, this information may be received from another device,such as a transmitter associated with a traffic signal light and/or fromanother vehicle which has made the determination. As such, the computersmay also determine when the traffic signal turns from green to yellowusing any of the examples above.

In one aspect the decision on whether to stop at a yellow traffic signalmay be based on how much braking the vehicle would need to exercise, inorder to come to a stop before the intersection. For example, if therequired deceleration to stop the vehicle in a given distance to atraffic intersection is larger than a threshold, the vehicle's computersmay determine that the vehicle should continue through the intersection.If not, the vehicle's computers may stop the vehicle before theintersection. While such logic works reasonably well for many scenarios,it can fail in situations when the autonomous vehicle (1) is alreadybraking (and thus braking a bit more would allow the autonomous vehicleto come to a stop), (2) is accelerating (and thus changing to hardbraking is unreasonable), or (3) the light turns red well beforereaching the intersection, but the autonomous vehicle decided to gothrough based on the above logic.

In order to address such issues, additional factors may be taken intoconsideration. For example, when approaching a traffic signal, theautonomous vehicle's computers may identify a time when the trafficsignal will turn red. For example, this time may be estimated by thecomputer by access pre-stored information regarding the length of time ayellow traffic signal will remain yellow. This information may be adefault estimation value, a measured value for the particular trafficsignal or intersection, or mathematically based on a speed limit for theroad on which the autonomous vehicle is currently traveling.Alternatively, this length of time or a future time when the light willturn from yellow to red may be received from another device, such as atransmitter associated with a traffic signal light and/or from anothervehicle which has made the determination.

The vehicle's computers may also determine a speed profile for thevehicle. This speed profile may be determined iteratively for a briefperiod of time in the future using any number of constraints including,but not limited to, road speed limit, curvature along trajectory of theautonomous vehicle, minimum and maximum acceleration the autonomousvehicle can execute, smoothness of acceleration, as well as othertraffic participants along the trajectory. For example, a speed profilemay be determined by analyzing all constraints for every second for thenext 20 seconds or so as cost functions, where each cost function wouldbe a lower value if the autonomous vehicle is in a desirable state and ahigher value otherwise. By summing each of the cost function values, thecomputers may select the speed profile having the lowest total costvalue. Non-linear optimization may allow for the determination of asolution in a short period of time. In addition, a speed profilecomputed in a previous iteration of the decision logic may also be usedto determine a current speed profile. This can save time for making thedecision more quickly as the speed profile usually does only changelittle if the overall traffic situation does not change abruptly.

Using the speed profile, the computers may estimate a future location ofthe autonomous vehicle at the time when the traffic signal will changefrom yellow to red. This time may be determined using the informationregarding the length of time a yellow traffic signal will remain yellowas well as the time when the computers determined that the trafficsignal changed from green to yellow. Then using this length of time andthe speed profile, the computers may estimate where the autonomousvehicle will be located when the traffic signal light will change fromyellow to red. Alternatively, if the vehicle's computer receivesinformation identifying a future time when a traffic signal light willturn from yellow to red from another device, this future time may beused to estimate where the autonomous vehicle will be located when thetraffic signal light will change from yellow to red.

The estimated future location may be determined using a referencelocation relative to the autonomous vehicle. For example, this referencelocation may be a most rear facing point on the autonomous vehicle, apoint in the center of a rear axle of the autonomous vehicle, point orplane of a rear bumper of the autonomous vehicle, etc. In some examples,the reference may be selected based upon a legal requirement, such as arequirement that a vehicle's rear wheels be within an intersectionbefore a traffic signal turns red. In such an example, the point orreference may include a point on one of the rear tires or a planeextending between the rear tires.

Using the estimated future location, the computers may determine whetherthe reference location will reach a certain location within theintersection when the traffic signal will change from yellow to red. Forexample, the detailed map information may define locations where theintersection starts and ends as well as threshold information.Alternatively, the location of the intersection may be determined by thevehicle's computer using sensor information, such as data from a cameraand/or laser, to detect typical features of an intersection such as awhite stop line, the location of traffic signals, or locations at whichone or more other roadways meeting with the roadway on which the vehicleis driving. This information may be used to identify a location defininga starting point for the intersection where the vehicle should stop,such as no further than the first traffic signal at an intersection. Ifthe estimated location of the vehicle is at least a threshold distanceinto the intersection and from the location defining the starting pointof the intersection, the autonomous vehicle may continue through theintersection. If not, the autonomous vehicle may stop.

The threshold distance may initially be a pre-determined distance.However, as the vehicle moves closer to the intersection, because of thepossibility of changes to the speed profile based on other factors (suchas other vehicles, etc.), this threshold may actually decrease as theautonomous vehicle approaches the intersection.

The aspects described herein thus allow for the autonomous vehicle'scomputers to make decisions about whether to enter an intersection inresponse to a yellow traffic signal. Again, such aspects also allow theautonomous vehicle to comply with legal requirements that require avehicle to have its rear-most axle within an intersection when a trafficsignal turns red.

EXAMPLE SYSTEMS

As shown in FIG. 1, a vehicle 100 in accordance with one aspect of thedisclosure includes various components. While certain aspects of thedisclosure are particularly useful in connection with specific types ofvehicles, the vehicle may be any type of vehicle including, but notlimited to, cars, trucks, motorcycles, busses, boats, airplanes,helicopters, lawnmowers, recreational vehicles, amusement park vehicles,farm equipment, construction equipment, trams, golf carts, trains, andtrolleys. The vehicle may have one or more computing devices, such ascomputing device 110 containing one or more processors 120, memory 130and other components typically present in general purpose computingdevices.

The memory 130 stores information accessible by the one or moreprocessors 120, including data 132 and instructions 134 that may beexecuted or otherwise used by the processor(s) 120. The memory 130 maybe of any type capable of storing information accessible by theprocessor(s), including a computing device-readable medium, or othermedium that stores data that may be read with the aid of an electronicdevice, such as a hard-drive, memory card, ROM, RAM, DVD or otheroptical disks, as well as other write-capable and read-only memories.Systems and methods may include different combinations of the foregoing,whereby different portions of the instructions and data are stored ondifferent types of media.

The data 132 may be retrieved, stored or modified by processor(s) 120 inaccordance with the instructions 134. For instance, although the claimedsubject matter is not limited by any particular data structure, the datamay be stored in computing device registers, in a relational database asa table having a plurality of different fields and records, XMLdocuments or flat files. The data may also be formatted in any computingdevice-readable format.

The instructions 134 may be any set of instructions to be executeddirectly (such as machine code) or indirectly (such as scripts) by theprocessor. For example, the instructions may be stored as computingdevice code on the computing device-readable medium. In that regard, theterms “instructions” and “programs” may be used interchangeably herein.The instructions may be stored in object code format for directprocessing by the processor, or in any other computing device languageincluding scripts or collections of independent source code modules thatare interpreted on demand or compiled in advance. Functions, methods androutines of the instructions are explained in more detail below.

The one or more processors 120 may be any conventional processors, suchas commercially available CPUs. Alternatively, the one or moreprocessors may be a dedicated device such as an ASIC or otherhardware-based processor, such as a field programmable gate array(FPGA). Although FIG. 1 functionally illustrates the processor(s),memory, and other elements of computing device 110 as being within thesame block, it will be understood by those of ordinary skill in the artthat the processor, computing device, or memory may actually includemultiple processors, computing devices, or memories that may or may notbe stored within the same physical housing. For example, memory may be ahard drive or other storage media located in a housing different fromthat of computing device 110. Accordingly, references to a processor orcomputing device will be understood to include references to acollection of processors or computing devices or memories that may ormay not operate in parallel.

Computing device 110 may have all of the components normally used inconnection with a computing device such as the processor and memorydescribed above, as well as a user input 150 (e.g., a mouse, keyboard,touch screen and/or microphone) and various electronic displays (e.g., amonitor having a screen, a small LCD touch-screen or any otherelectrical device that is operable to display information). In thisexample, the vehicle includes an internal electronic display 152. Inthis regard, internal electronic display 152 may be located within acabin of vehicle 100 and may be used by computing device 110 to provideinformation to passengers within the vehicle 100.

In one example, computing device 110 may be an autonomous drivingcomputing system incorporated into vehicle 100. The autonomous drivingcomputing system may capable of communicating with various components ofthe vehicle as needed in order to control the vehicle in fullyautonomous (without input from a driver) as well as semiautomonus (someinput from a driver) driving modes.

As an example, FIG. 2 depicts an interior design of a vehicle havingautonomous, semiautonomous, and manual (continuous input from a driver)driving modes. In this regard, the autonomous vehicle may include all ofthe features of a non-autonomous vehicle, for example: a steeringapparatus, such as steering wheel 210; a navigation display apparatus,such as navigation display 215 (which may be a part of electronicdisplay 152); and a gear selector apparatus, such as gear shifter 220.The vehicle may also have various user input devices 140 in addition tothe foregoing, such as touch screen 217 (again, which may be a part ofelectronic display 152), or button inputs 219, for activating ordeactivating one or more autonomous driving modes and for enabling adriver or passenger 290 to provide information, such as a navigationdestination, to the computing device 110.

Returning to FIG. 1, when engaged, computer 110 may control some or allof these functions of vehicle 100 and thus be fully or partiallyautonomous. It will be understood that although various systems andcomputing device 110 are shown within vehicle 100, these elements may beexternal to vehicle 100 or physically separated by large distances.

In this regard, computing device 110 may be in communication varioussystems of vehicle 100, such as deceleration system 160, accelerationsystem 162, steering system 164, signaling system 166, navigation system168, positioning system 170, and perception system 172, such that one ormore systems working together may control the movement, speed,direction, etc. of vehicle 100 in accordance with the instructions 134stored in memory 130. Although these systems are shown as external tocomputing device 110, in actuality, these systems may also beincorporated into computing device 110, again as an autonomous drivingcomputing system for controlling vehicle 100.

As an example, computing device 110 may interact with decelerationsystem 160 and acceleration system 162 in order to control the speed ofthe vehicle. Similarly, steering system 164 may be used by computingdevice 110 in order to control the direction of vehicle 100. Forexample, if vehicle 100 configured for use on a road, such as a car ortruck, the steering system may include components to control the angleof wheels to turn the vehicle. Signaling system 166 may be used bycomputing device 110 in order to signal the vehicle's intent to otherdrivers or vehicles, for example, by lighting turn signals or brakelights when needed.

Navigation system 168 may be used by computing device 110 in order todetermine and follow a route to a location. In this regard, thenavigation system 168 and/or data 132 may store map information, e.g.,highly detailed maps identifying the shape and elevation of roadways,lane lines, intersections, crosswalks, speed limits, traffic signals,buildings, signs, real time traffic information, vegetation, or othersuch objects and information.

FIG. 3 is an example of detailed map information 300 for a section ofroadway including an intersection 302. In this example, the detailed mapinformation 300 includes information identifying the shape, location,and other characteristics of lane lines 310, 312, 314, traffic signals320, 322, 324, 326, crosswalks 330, 332, 334, 336. Although the examplesherein relate to a simplified three-light, three-state traffic signals(e.g, a green light state indicating that a vehicle can proceed throughan intersection, a yellow light state indicating a transition statewhere the vehicle can proceed with caution but may need to stop at anintersection, and a red light state indicating that the vehicle shouldstop before the intersection, etc.), other types of traffic signals,such as those for right or left turn only lanes may also be used.

In addition, the detailed map information includes a network of rails340, 342, 344, which provide the vehicle's computer with guidelines formaneuvering the vehicle so that the vehicle follows the rails and obeystraffic laws. As an example, a vehicle's computer may maneuver thevehicle from point A to point B (two fictitious locations not actuallypart of the detailed map information) by following rail 340,transitioning to rail 342, and subsequently transitioning to rail 344 inorder to make a left turn at intersection 302.

The detailed map information 300 may also include a number of markers350, 352, 354, and 356 which indicate the beginning of a trafficintersection or the point by which the vehicle must be stopped iftraffic signal corresponding to the vehicle's current lane were to besignaling a red light. For simplicity, additional details, such asadditional traffic signals and rails, of the intersection 302 are notshown. Although the example herein is shown pictorially for a right-lanedrive location, the detailed map information may be stored in any numberof different ways including databases, roadgraphs, etc. and may includemaps of left-lane drive locations as well.

Positioning system 170 may be used by computing device 110 in order todetermine the vehicle's relative or absolute position on a map or on theearth. For example, the position system 170 may include a GPS receiverto determine the device's latitude, longitude and/or altitude position.Other location systems such as laser-based localization systems,inertial-aided GPS, or camera-based localization may also be used toidentify the location of the vehicle. The location of the vehicle mayinclude an absolute geographical location, such as latitude, longitude,and altitude as well as relative location information, such as locationrelative to other cars immediately around it which can often bedetermined with less noise that absolute geographical location.

The positioning system 170 may also include other devices incommunication with computing device 110, such as an accelerometer,gyroscope or another direction/speed detection device to determine thedirection and speed of the vehicle or changes thereto. By way of exampleonly, an acceleration device may determine its pitch, yaw or roll (orchanges thereto) relative to the direction of gravity or a planeperpendicular thereto. The device may also track increases or decreasesin speed and the direction of such changes. The device's provision oflocation and orientation data as set forth herein may be providedautomatically to the computing device 110, other computing devices andcombinations of the foregoing.

The perception system 172 also includes one or more components fordetecting and performing analysis on objects external to the vehiclesuch as other vehicles, obstacles in the roadway, traffic signals,signs, trees, etc. For example, the perception system 172 may includelasers, sonar, radar, one or more cameras, or any other detectiondevices which record data which may be processed by computing device110. In the case where the vehicle is a small passenger vehicle such asa car, the car may include a laser mounted on the roof or otherconvenient location as well as other sensors such as cameras, radars,sonars, and additional lasers. The computing device 110 may control thedirection and speed of the vehicle by controlling various components. Byway of example, if the vehicle is operating completely autonomously,computing device 110 may navigate the vehicle to a location using datafrom the detailed map information and navigation system 168. Computingdevice 110 may use the positioning system 170 to determine the vehicle'slocation and perception system 172 to detect and respond to objects whenneeded to reach the location safely. In order to do so, computing device110 may cause the vehicle to accelerate (e.g., by increasing fuel orother energy provided to the engine by acceleration system 162),decelerate (e.g., by decreasing the fuel supplied to the engine or byapplying brakes by deceleration system 160), change direction (e.g., byturning the front or rear wheels of vehicle 100 by steering system 164),and signal such changes (e.g. by lighting turn signals of signalingsystem 166).

FIG. 4 is an example external view of vehicle 100 described above. Asshown, various components of the perception system 172 may be positionedon or in the vehicle 100 in order to better detect external objectswhile the vehicle is being driven. In this regard, one or more sensors,such as laser range finders 410 and 412 may be positioned or mounted onthe vehicle. As an example, the one or more computing devices 110 (notshown) may control laser range finder 410, e.g., by rotating it 180degrees. In addition, the perception system may include one or morecameras 420 mounted internally on the windshield of vehicle 100 toreceive and analyze various images about the environment. In addition tothe laser range finder 410 is positioned on top of perception system 172in FIG. 4, and the one or more cameras 420 mounted internally on thewindshield, other detection devices, such as sonar, radar, GPS, etc.,may also be positioned in a similar manner.

The one or more computing devices 110 may also include features such astransmitters and receivers that allow the one or more devices to sendand receive information to and from other devices. For example, the oneor more computing devices may determine information about the currentstatus of a traffic signal light as well as information about when thestatus of the traffic signal light changes (from green to yellow to redto green). The one or more computing devices may send this informationto other computing devices associated with other vehicles. Similarly,the one or more computing devices may receive such information fromother computing devices. For example, the one or more computing devicesmay receive information about the current status of a traffic signallight as well as information about when the status of the traffic signallight changes from one or more other computing devices associated withother autonomous or non-autonomous vehicles. As another example, the oneor more computing devices may receive such information with devicesassociated with the traffic signal lights. In this regard, some trafficsignal lights may include transmitters that send out information aboutthe current status of a traffic signal light as well as informationabout when the status of the traffic signal light changes.

This information may be sent and received via any wireless transmissionmethod, such as radio, cellular, Internet, World Wide Web, intranets,virtual private networks, wide area networks, local networks, privatenetworks using communication protocols proprietary to one or morecompanies, Ethernet, WiFi and HTTP, and various combinations of theforegoing. Such communication may be facilitated by any device capableof transmitting data to and from other computers, such as modems andwireless interfaces.

EXAMPLE METHODS

In addition to the operations described above and illustrated in thefigures, various operations will now be described. It should beunderstood that the following operations do not have to be performed inthe precise order described below. Rather, various steps can be handledin a different order or simultaneously, and steps may also be added oromitted.

As noted above, a vehicle's one or more computing devices may maneuverthe vehicle using the various systems described above. For example, FIG.5 depicts a section of roadway 500 including an intersection 502.Vehicle 100 is approaching intersection 502 and may be controlled, forexample by one or more computing device 110 in an autonomous drivingmode as described above.

In this example, intersection 502 corresponds to the intersection 302 ofthe detailed map information 300. In this regard, lane lines 510, 512,and 514 correspond to the shape, location, and other characteristics oflane lines 310, 312, and 314, respectively. Similarly crosswalks 530,532, 534, and 536 correspond to the shape, location, and othercharacteristics of crosswalks 330, 332, 334, and 336, respectively andtraffic signal 526 corresponds to the shape, location, and othercharacteristics of traffic signal 326. For simplicity, only a singletraffic signal 526 is shown, though other traffic signals correspondingto the shape, location, and other characteristics of traffic signals320, 322, and 324 may also exist.

An autonomous vehicle's one or more computing devices may identify thestate of a traffic signal using any known techniques. For example, usinga combination of sensor data and the expected location of trafficsignals in the detailed map information, the computers may estimate anapproximate location of a traffic signal related to the location of avehicle. For example, returning to FIG. 3, when a vehicle is at point A(corresponding to the location of vehicle 100 in FIG. 5), the vehicle'scomputer may access the detailed map information in order to determinethat traffic signals 320, 322, 324, and 326 are all relevant to theposition of the vehicle 100. Thus, all other traffic signals, such asthose that would be relevant to a vehicle positioned at points B or C inFIG. 3 (two additional fictitious locations not actually part of thedetailed map information) may be ignored. Then using templates, imagematching color detection in images, etc. the computers may determine thestate of these traffic signals (red light, yellow light, or greenlight). Alternatively, the locations of traffic signals may be estimatedsolely from the sensor data, and without reference to detailed mapinformation. As another example, the locations of traffic signals may bereceived from another device, such as a transmitter associated with atraffic signal light and/or from another vehicle which has made thedetermination.

The one or more computing devices 110 may also identify when the trafficsignals change state. For example, the vehicle's one or more computingdevices 110 may monitor the state of these traffic signals over a briefperiod of time to determine when the traffic signal changes state. Inthe example of FIG. 5, the vehicle's one or more computing devices mayestimate a time when the traffic signal changes states based on theinformation used to identify the state of the traffic signals. Forexample, using timestamp information associated with the images capturedby the vehicle's cameras several times per second, the vehicle's one ormore computing devices may estimate a time when a traffic signal changesstate from one light color to another, such as from a green light stateto a yellow light state (green to yellow). This information may then beused to determine whether to stop the vehicle (because the trafficsignal light may eventually turn red) or continue through theintersection without stopping. Alternatively, the time when a trafficsignal changes state from one light color to another may be receivedfrom another device, such as a transmitter associated with a trafficsignal light and/or from another vehicle which has made thedetermination.

The vehicle's one or more computing devices may determine whether tocontinue through or to stop at an intersection when a relevant trafficsignal is in the yellow light state based on how much braking powerwould be required to stop the vehicle by the intersection. For examplereturning to FIG. 3, if “d” is the remaining distance between point Aand marker 350 (which indicates the beginning of intersection 302), and“v” is the current speed of vehicle 100 then the required deceleration“a” to stop is given as a =0.5 v{circumflex over ( )}2/d. In thisexample, if v is in meters per second and d is in meters, then thedeceleration a will be in meters per second squared. The vehicle's oneor more computing devices 110 may compare the deceleration a to athreshold value to determine whether the vehicle should go through theintersection. The threshold value may be selected based upon acomfortable breaking power for a passenger of the vehicle. Too muchdeceleration may be scary for a passenger (e.g. it would be as if adriver were to “slam” on the breaks in order to stop at anintersection). Thus, if this deceleration a is larger than thethreshold, the vehicle's one or more computing devices 110 may allow thevehicle to continue through the intersection, and if not, the vehicle'sone or more computing devices may stop the vehicle.

In other examples, additional factors may be taken into considerationwhen the vehicle's one or more computing devices are determining whetherto continue through an intersection when a relevant traffic signal is ina yellow light state. For example, when approaching a traffic signal,the autonomous vehicle's one or more computing devices 110 may identifya time when the traffic signal light will change from the yellow lightstate to the red light state (yellow to red).

In one instance, the computer may first identify a length of time that atraffic signal will remain in the yellow light state. This may includeaccessing information regarding the length of time a yellow trafficsignal will remain yellow. This information may be a default estimationvalue (e.g., always 2.5 seconds), a measured value for the particulartraffic signal or intersection (e.g., specific to traffic signal 326),or mathematically based on a speed limit for the road on which thevehicle 100 is currently traveling. As an example, on a 25 mile per hourroad, traffic signal lights may have a period yellow light on the orderof less than 4 seconds, whereas on a 35 mile per hour road, the trafficsignal lights may have a longer period yellow light, such as 4 or moreseconds. Alternatively, this information may be received from anotherdevice, such as a transmitter associated with a traffic signal lightand/or from another vehicle which has made the determination.

Based on the identified length of time, the vehicle's one or morecomputing devices may identify a time when the traffic signal will turnfrom the yellow light state to a red light state (yellow to red). Thismay be done by simply adding the identified length of time to theestimated time when the traffic signal turned from the green light stateto the yellow light state. Alternatively, a future time when the trafficsignal will turn from the yellow light state to a red light state may bereceived from another device, such as a transmitter associated with atraffic signal light and/or from another vehicle which has made thedetermination.

The vehicle's one or more computing devices may also determine a speedprofile for the vehicle. A speed profile may describe an expected futurespeed and in some cases an expected future acceleration and decelerationfor a plurality of different times in the future. As noted above, thisspeed profile may be determined iteratively for a brief period of time(e.g., every second, or more or less, for the next 20 seconds, or moreor less). As an example of a simple speed profile which ignores morecomplex factors, the vehicle's one or more computing devices may assumethat the vehicle's current speed will continue (e.g., there will be noacceleration or deceleration) over the brief period or use the vehicle'scurrent speed and acceleration and deceleration to estimate thevehicle's future speed.

A speed profile may also be determined using any number of additionalconstraints including, but not limited to, road speed limit, curvaturealong trajectory of the autonomous vehicle, minimum and maximumacceleration the autonomous vehicle can execute, smoothness ofacceleration, as well as other traffic participants along thetrajectory. For example, a speed profile may be determined by analyzingall such constraints for every second for the next 20 seconds or so ascost functions, where each cost function would be a lower value if theautonomous vehicle is in a desirable state and a higher value otherwise.By summing each of the cost function values, the computers may selectthe speed profile having the lowest total cost value. Non-linearoptimization may allow for the determination of a solution in a shortperiod of time.

In addition, a speed profile computed in a previous iteration of thedecision logic may also be used to determine a current speed profile.This can save time for making the decision more quickly as the speedprofile usually does only change little if the overall traffic situationdoes not change abruptly.

Using the speed profile, the vehicle's one or more computing devices mayestimate a future location of the autonomous vehicle at the time whenthe traffic signal will change from the yellow light state to the redlight state. For example, using the simplest speed profile (where thespeed of the vehicle does not change) the future location may bedetermined by multiplying the speed of the vehicle by the differencebetween the current time and the estimated time when the traffic signalwill change from the yellow light state to the red light state.Alternatively, if the vehicle's computer receives informationidentifying a future time when a traffic signal light will turn fromyellow to red from another device, this future time may be used toestimate where the autonomous vehicle will be located when the trafficsignal light will change from yellow to red.

Of course, where the speed profile is more complex, estimating thefuture location may be simplified by finding the vehicle's average speeduntil the time when the traffic signal will change from the yellow lightstate to the red light state and multiplying this value by thedifference between the current time and the estimated time when thetraffic signal will change from the yellow light state to the red lightstate. Again, if the vehicle's computer receives information identifyinga future time when a traffic signal light will turn from yellow to redfrom another device, this future time may be used to estimate where theautonomous vehicle will be located when the traffic signal light willchange from yellow to red. FIG. 6 is an example of an estimated futurelocation of vehicle 100 (shown in dashed line as it is an estimate) atthe time when traffic signal 526 changes from the yellow light state tothe red light state relative to the section of roadway 500.

The estimated future location may be determined using a referencelocation relative to the autonomous vehicle. For example, this referencelocation may be a most rear facing point on the vehicle 100, a point inthe center or along a plane of a rear axle of the vehicle as indicatedby arrow 622, point or plane of a rear bumper of the vehicle asindicated by arrow 620, etc. In some examples, the reference may beselected based upon a legal requirement, such as a requirement that avehicle's rear wheels be within an intersection before a traffic signalturns red. In such an example, the point or reference may include apoint on one of the rear tires or a plane extending between the reartires as indicated by arrow 620.

The vehicle's one or more computing devices may also identify a startingpoint of an intersection associated with the relevant traffic signal.For example, as noted above, the detailed map information may includemarkers that define locations where the intersection starts and ends aswell as threshold information. The relevant marker for an intersectionmay be the marker that the vehicle will pass before an intersectionalong a particular rail. Returning to FIG. 3, when traveling along rail340 from point 1 towards intersection 302, the vehicle will first passthrough marker 350. Thus, marker 350 may be the relevant marker forintersection 302.

Alternatively, the location of the intersection may be determined by thevehicle's computer using sensor information, such as data from a cameraand/or laser, to detect typical features of an intersection such as awhite stop line, the location of traffic signals, or locations at whichone or more other roadways meeting with the roadway on which the vehicleis driving. This information may be used to identify a location defininga starting point for the intersection where the vehicle should stop,such as no further than the first traffic signal at an intersection.

The vehicle's one or more computing devices may determine whether thevehicle, and in some cases the reference location of the vehicle, willbe at least a threshold distance passed the starting point when thetraffic signal will change from the yellow light state to the red lightstate. For example, the threshold distance may initially be apre-determined distance, such as 5 meters or more or less, selectedbased upon the stopping power of the vehicle as in the example describedabove. For example, FIG. 7 is another example of the detailed mapinformation 300 but includes reference point 710 indicating a thresholddistance Th along the rail 340 passed marker 350.

Based on whether the estimated future location of the vehicle isdetermined to be at least the threshold distance past the startingpoint, the vehicle's one or more computing devices, may determinewhether the vehicle should continue through the intersection. If theestimated future location is at least a threshold distance passed thelocation of the relevant marker (or at least a threshold distance intothe intersection) the vehicle's one or more computing devices 110 mayallow the vehicle to continue through the intersection. If not, the oneor more computing devices 110 may stop the vehicle by the location ofthe marker.

For example, FIG. 8 is an example of the section of roadway 500including the future estimated location of vehicle 100 at the time whentraffic signal 526 changes from the yellow light state to the red lightstate. However, this example also includes the location of marker 350and reference point 710 of FIG. 7. Here, the future estimated locationof vehicle 100 (even relative to the planes of arrows 620 and 622) ispassed the reference point 710. In this regard, the vehicle's one ormore computing devices may allow the vehicle to continue through theintersection. Similarly, if vehicle 100 were to not pass the location ofreference point 710 by the time that the when traffic signal 526 changesfrom the yellow light state to the red light state, the vehicle's one ormore computing devices may being the vehicle to a atop at or before thelocation of marker 350.

As the vehicle becomes closer to the intersection, because of thepossibility of changes to the speed profile based on other factors (suchas other vehicles, etc.), the threshold may actually decrease. Forexample, as shown in FIG. 9, as vehicle 100 approaches the intersection502, the threshold distance Th may decrease to Th′. Thus, when vehicle100 moves from the location of point E to the location of point F, thereference point 710 may be relocated to the location of reference point910. Thus, as the vehicle 100 moves closer to the intersection 502, thevehicle need not be as far into the intersection 502 in order for thevehicle's one or more computing devices to allow the vehicle to continuethrough the intersection.

FIG. 10 is an example flow diagram 1000 which depicts some of theaspects described above which may be performed by one or more computingdevices such as one or more computing devices 110 of vehicle 100. Inthis example, the information identifying a time when a traffic signallight turned from green to yellow is received at block 1010. A length oftime that the traffic signal light will remain yellow is identified atblock 1020. A time when the traffic signal light will turn from yellowto red based on the time when a traffic signal light turned from greento yellow and a length of time that the traffic signal light will remainyellow is estimated at block 1030. A location of a vehicle at theestimated time that the traffic signal light will turn from yellow tored is estimated at block 1040. A starting point of an intersectionassociated with the traffic signal light based on detailed mapinformation is identified at block 1050. Whether the estimated locationof the vehicle will be at least a threshold distance past the startingpoint is determined at block 1060. When the estimated location of thevehicle is determined to be at least a threshold distance past thestarting point, the vehicle should continue through the intersection isdetermined at block 1070.

In some examples, the traffic signal light may turn red before theestimated time that the traffic signal light will turn from red toyellow. This may be for various reasons such as a misclassification ofthe light color when determining the status of a traffic signal light,incorrectly identifying a length of time that a traffic signal willremain in the yellow light state, or simply because the timing of theparticular traffic signal light includes some anomaly. In such a case,the vehicle's computing device may make the direct computation ofwhether the vehicle can be stopped in time without going too far intothe intersection (e.g., not more than 1 meter or more or less for safetyreasons), for example, using the braking power example described above.

Unless otherwise stated, the foregoing alternative examples are notmutually exclusive, but may be implemented in various combinations toachieve unique advantages. As these and other variations andcombinations of the features discussed above can be utilized withoutdeparting from the subject matter defined by the claims, the foregoingdescription of the embodiments should be taken by way of illustrationrather than by way of limitation of the subject matter defined by theclaims. In addition, the provision of the examples described herein, aswell as clauses phrased as “such as,” “including” and the like, shouldnot be interpreted as limiting the subject matter of the claims to thespecific examples; rather, the examples are intended to illustrate onlyone of many possible embodiments. Further, the same reference numbers indifferent drawings can identify the same or similar elements.

1. A method of controlling a vehicle having an autonomous driving mode,the method comprising: determining, by one or more processors, adistance from the vehicle to an intersection; determining, by the one ormore processors, a first state of a traffic signal when the vehicle isthe distance from the intersection; determining, by the one or moreprocessors, a time when the vehicle will enter the intersection based ona current speed of the vehicle and the distance; determining, by the oneor more processors, a second state of the traffic signal at the timewhen the vehicle will enter the intersection based on the determinedfirst state; and controlling, by the one or more processors, the vehiclethrough the intersection in the autonomous driving mode based on thedetermined second state.
 2. The method of claim 1, wherein in thedetermined first state includes the traffic signal having an illuminatedyellow light.
 3. The method of claim 2, wherein in the determined secondstate includes the traffic signal having an illuminated red light, andwherein controlling the vehicle through the intersection includesstopping the vehicle before the intersection.
 4. The method of claim 1,wherein the determined second state includes the traffic signal havingan illuminated yellow light, and wherein controlling the vehicle throughthe intersection includes proceeding through the intersection withoutstopping the vehicle.
 5. The method of claim 1, wherein controlling thevehicle through the intersection is further based on how much brakingpower would be required to stop the vehicle by the intersection based onthe distance and the current speed of the vehicle.
 6. The method ofclaim 5, further comprising, comparing the braking power to a thresholdvalue to determine whether the vehicle should proceed through theintersection without stopping, and wherein controlling the vehiclethrough the intersection is further based on the determination ofwhether the vehicle should proceed through the intersection withoutstopping.
 7. The method of claim 6, wherein when the comparisonindicates that the braking power is greater than the threshold value,controlling the vehicle through the intersection includes proceedingthrough the intersection without stopping the vehicle.
 8. The method ofclaim 6, wherein when the comparison indicates that the braking power isless than the threshold value, controlling the vehicle through theintersection includes stopping the vehicle before the intersection. 9.The method of claim 1, further comprising identify a time when thetraffic signal will change state, and wherein controlling the vehiclethrough the intersection is further based on the time when the trafficsignal will change state.
 10. The method of claim 1, wherein thedetermined first state is the same as the determined second state. 11.The method of claim 1, wherein the determined second state is differentfrom the determined first state.
 12. The method of claim 1, furthercomprising, identifying a length of time that the traffic signal willremain in the determined first state, and wherein controlling thevehicle through the intersection is further based on the length of time.13. The method of claim 12, wherein the length of time is based on adefault estimation time.
 14. The method of claim 12, wherein the lengthof time is based on a speed limit for a road on which the vehicle istraveling.
 15. The method of claim 12, wherein the length of time isreceived from a device remote from the vehicle.
 16. The method of claim12, wherein determining the determined second state is further based onthe length of time.
 17. A system for controlling a vehicle having anautonomous driving mode, the system comprising one or more processorsconfigured to: determine a distance from the vehicle to an intersection;determine a first state of a traffic signal when the vehicle is thedistance from the intersection; determine a time when the vehicle willenter the intersection based on a current speed of the vehicle and thedistance; determine a second state of the traffic signal at the timewhen the vehicle will enter the intersection based on the determinedfirst state; and control the vehicle through the intersection in theautonomous driving mode based on the determined second state.
 18. Thesystem of claim 17, wherein the determined first state includes thetraffic signal having an illuminated yellow light, and whereincontrolling the vehicle through the intersection includes proceedingthrough the intersection without stopping the vehicle.
 19. The system ofclaim 17, wherein the one or more processors are configured to controlthe vehicle through the intersection further based on how much brakingpower would be required to stop the vehicle by the intersection based onthe distance and the current speed of the vehicle.
 20. The system ofclaim 18, further comprising the vehicle.